Output correction for visual projection devices

ABSTRACT

Dedicated projectors and devices with projection abilities such as digital cameras or camcorders have projections that are corrected to compensate for irregularities in the surfaces receiving the projection. The image data being projected is compensated to account for the irregularities by observing the irregularities with a camera to produce image data and creating the compensation based on that image data. Irregularities of the projection receiving location including angular relationships to the projector causing keystoning, noisy surfaces with reflectivity or absorption, color patterns, non-planar regions, intervening objects, and the like may be accounted for during the compensation. The projection of the image itself may be utilized to capture the result of projecting onto the irregularities. Projection of target grids, such as using infrared, may be used to capture the result of projecting onto the irregularities. The captured image may be processed to produce a compensated image to be projected.

TECHNICAL FIELD

Embodiments are related to the projection of visual displays. Moreparticularly, embodiments are related to correcting the output ofprojection devices.

BACKGROUND

Projectors project a visual image onto a surface, typically a projectorscreen that provides a nearly ideal projection receiving surface. Theprojector screen typically has a plane white color, a suitablereflectivity for viewing the projected image when dim room lighting ispresent, and is a flat vertical surface that does not distort theprojected image so long as the projector is properly aligned. However,it is not always possible to ideally position the projector, and attimes, improper alignment occurs and distortion such as keystoningresults. Another drawback occurs when a presenter or other object ispresent between the projector and the projection screen. The projectionis distorted at the point where the projection output reaches the personor object rather than the projector screen. Furthermore, the projectionis bothersome to the person when facing the projector.

With projection devices being miniaturized and/or combined with otherdevices, such as placing projectors within digital cameras, camcorders,cell phones, and other portable digital devices so that individuals caneasily share images, the need to project an image may arise at any timeand place. Therefore, the surface to receive the projection output maybe anything from a table top to a person's clothing. Thus, the surfacereceiving the projection output may be far from the ideal of theprojector screen. Therefore, the resulting image appearing on thesurface receiving the projection output may be distorted in many ways,due to non-planar surfaces, dynamic surfaces, oddly colored and/ortextured surfaces, and so forth. Furthermore, the alignment of theprojection output to the surface may be angular and result inkeystoning. In these situations, the appearance of the projected imagemay be less than desirable.

SUMMARY

Embodiments address issues such as these and others by providingcorrection of the projection output by capturing images of the surfacereceiving the projection. The captured images may be captures of thedesired image being projected onto the surface to reveal the distortionsproduced by the irregularities of the surface. The captured images mayalternatively be captures of a target such as an infrared grid thatreveal the distortions produced by the irregularities. The capturedimages are then used to calculate corrections to be applied to the imagedata that will compensate for the irregularities of the projectionreceiving surface.

Embodiments provide a device that includes a housing and a projectionoutput within the housing producing a projected output that extends to afirst position beyond the housing. The device includes a camera withinthe housing that has a fixed relationship relative to the projectionoutput and that captures an image from the first position. The devicefurther includes a processor within the housing and in communicationwith the projection output and the camera. The processor provides asource image to the projection output, receives the captured image fromthe camera, and compares the captured image relative to the source imagein relation to the fixed relationship between the camera and theprojection output to determine at least one difference. The processoralso creates an image based on the at least one difference, and providesthe created image to the projection output in placed of the sourceimage.

Embodiments provide a computer readable medium containing instructionsthat perform acts that include projecting a reference target onto afirst location. The acts further include capturing image data of thefirst location while the target is being projected onto the firstlocation and comparing the captured image data to the reference targetto detect at least one difference. The acts also include generatingcompensation data based on the at least one difference, applying thecompensation data to image data to produce compensated image data, andprojecting a compensated image corresponding to the compensated imagedata onto the first location.

Embodiments provide a method of projecting an image that involvesobtaining reference image data and producing a reference display signalfrom the reference image data, where the reference display signal isprojected onto a first location, the first location being a dynamicsurface. The method involves capturing first image data of the firstlocation while the reference display signal is being projected onto thefirst location while the first location is in a first state. The methodinvolves comparing the captured first image data to the reference imagedata to detect at least one first difference, generating firstcompensation data based on the at least one first difference, andapplying the first compensation data to the reference image data toproduce compensated first image data. The method involves producing afirst compensated display signal from the compensated image data, thefirst compensated display signal being projected onto the firstlocation. The method involves capturing second image data of the firstlocation while the first compensated display signal is being projectedonto the first location while the first location is in a second statedifferent than the first state. The method involves comparing thecaptured second image data to the reference image data to detect atleast one second difference, generating second compensation data basedon the at least one second difference, and applying the secondcompensation data to the reference image data to produce compensatedsecond image data. The method further involves producing a secondcompensated display signal from the compensated second image data, thesecond compensated display signal being projected onto the firstlocation.

Other systems, methods, and/or computer program products according toembodiments will be or become apparent to one with skill in the art uponreview of the following drawings and detailed description. It isintended that all such additional systems, methods, and/or computerprogram products be included within this description, be within thescope of the present invention, and be protected by the accompanyingclaims.

DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a device according to various embodiments projecting animage onto a surface having irregularities that distort the projectedimage.

FIG. 2 shows a device according to various embodiments projecting animage onto a surface having irregularities where the image iscompensated to reduce the effect of the irregularities.

FIG. 3 shows a device according to various embodiments projecting animage onto a surface having a person between the projector and thesurface where the person distorts the projected image.

FIG. 4 shows a device according to various embodiments projecting animage onto a surface having a person between the projector and thesurface where the image is compensated to reduce the effect ofprojecting onto the person.

FIG. 5 shows a device according to various embodiments projecting onto asurface having a gesturing hand present between the surface and theprojector.

FIG. 6 shows a device according to various embodiments projecting onto asurface after having manipulated an image for projecting the image inaccordance with a detected gesture.

FIG. 7A shows an exemplary configuration of components of a deviceaccording to various embodiments where an image is projected and thencaptured for correction.

FIG. 7B shows an exemplary configuration of components of a deviceaccording to various embodiments where a target is projected and thencaptured to correct an image.

FIG. 8 shows an example of logical operations that may be performed by adevice according to various embodiments in order to project an image andcapture the image for correction.

FIG. 9 shows an example of logical operations that may be performed by adevice according to various embodiments to implement corrections to animage.

FIG. 10 shows an example of logical operations that may be performed bya device according to various embodiments in order to project a targetand capture the target to correct an image.

DETAILED DESCRIPTION

Embodiments provide for the correction of projection outputs tocompensate for irregularities in the surface(s) receiving the projectedimage. An image of the surface is captured, where a projected image ortarget is present on the surface. From the image of the projected imageor target appearing on the surface, compensation for the surface can becalculated and applied to the image data. The compensation allows theimage data to be manipulated such that when projected onto the surface,the effects of the irregularities in the surface are decreased orotherwise changed.

FIG. 1 shows an example of a device 102 according to variousembodiments. The device 102 may be a portable digital device, such as adigital camera, digital camcorder, mobile telephone, personal digitalassistant, and the like. The device 102 may alternatively be a dedicatedprojection device, such as a full scale projector.

The device 102 includes a housing 103 within which sub-devices arepresent. The sub-devices may include a camera 106 as well as aprojection output 104. As shown in this example, the projection output104 is projecting an image 108 onto a table top 110. As the projectionoutput 104 is projecting at an angle relative to the table top 110, theprojected image 108 is keystoned. Furthermore, the projected image 108partially overlaps with a coin 112 also sitting on the table top 110which presents an additional distortion. In addition to that, the tabletop 110 has a prominent surface ornamentation such as wood grain orother streaks that pass through the location where the projected image108 appears so as to further distort the projected image 108.

According to exemplary embodiments, the camera 106 captures an image ofthe same location where the projection output 104 is directed. Thecamera 106 maintains a fixed relation to the projection output 104within the housing 103 such that the images produced from the camera 106have an expected format. For example, the camera 106 may experiencenearly the same amount of keystone as the projection output 104 but inreverse, so that if the projection image 108 is keystoned, it appearsless keystoned to the camera 106. Therefore, as discussed below thedevice 102 may apply this known condition when analyzing the imagecaptured by the camera 106 relative to the reference image being sent tothe projection output 104.

For example, to eliminate keystoning of the projection image 108 from avantage point directly over the image 108 may call for a predefinedamount of keystoning to appear in the image that is captured by thecamera 106 where the camera 106 has a vantage point other than directlyover the image being projected. Because the camera relationship 106 isknown relative to the position of the projection output 104, the amountof keystoning present in the camera image for an image that is actuallynot keystoned when viewed from directly above can be defined withinkeystone removal logic of the device 102. While the device 102 may becalibrated for removing keystoning for other vantage points, the vantagepoint directly over the image may be a common choice for calibrationbecause that mimics the appearance of an actual photograph lying on thetable top 110. In other words, individuals may expect there to be somenaturally occurring keystone when viewing the image from an angle butmay expect there to be no keystone when viewing the image directly fromabove.

Furthermore, the image captured by the camera 106 shows a shift in theimage due to the vertical change in the surface resulting from the coin112. Upon correcting the keystone of the projection output andaccounting for the known keystone of the camera 106, a pixel by pixeloverlay of the captured image relative to the reference image may bedone in memory of the device 102 to perceive the shift. The reverse ofthe shift may then be computed. As discussed above, different referencevantage points may be used to calibrate the correction for such a shift.For example, from a vantage point directly over the image, the shift maybe less than as perceived by the camera 106, such that the compensationis calibrated to correct for the shift by an amount less than isnecessary to eliminate the shift from the perspective of the camera 106.

In addition to correcting for such shifts due to variation in the planeof the projection receiving surface, the device 102 may also compensatefor the grain or other streaks present on the table top 110 that appearwithin the project image 108. Upon creating the pixel by pixel overlayin memory between the image captured by the camera 106 and the referenceimage being projected, the device 102 may also account for differencesin the intensity, color, hue, and other visual characteristics at eachpixel. Thus, if a pixel that should be blue appears green, then it canbe computed that a yellow contribution is coming from a background atthe point where that pixel appears.

Compensation may be computed to adjust the pixel color being output sothat when that pixel color appears on the background that is providingthe yellow contribution, it appears bluer and less green. While theprecise colors of the reference image may not always be possible on abackground surface of colors other than white, this pixel by pixelapproach may bring the colors closer to those that are present in theactual image data and may also improve upon the consistency of thecolors of a compensated projected image 108′. So, for example, if theblue discussed above is a different shade than a blue of the actualimage, the blue of the compensated image may be very close to otherareas of the image containing that blue and may also be closer to theblue of the actual image than to the unintended green that initiallyappeared at that pixel location.

Rather than capture the projected image 108 with the camera 106, thedevice may alternatively project a target grid onto the location wherethe projection image 108 is directed. Target grids are used forauto-focusing purposes with conventional cameras. As one example, twoslightly offset fixed pattern emitters, using infrared, visible light,or other spectrum, may be provided to project a target grid, one emitterprojecting a ‘=’ pattern and the other emitter projecting a ‘∥’ pattern.The image returned from a flat surface would be like a tic-tac-toepattern, but at angles, the distortion of the lines relative to eachother, and the shadows and other visual cues will allows a surface mapto be quickly perceived by the device 102. As another example, a singleemitter may be used to project the same or similar patterns such as the‘+’ pattern as shown below in FIG. 7B.

This target grid may be projected and captured by the camera 106 tocompute corrections for keystone, shifts in the surface, variations indepth of the field of view, and so forth by determining differences inthe reference grid relative to the captured grid. This target grid maybe projected by a target grid source before the projection image 108 isprojected so as to determine the compensation before presentation of theimage begins. Alternatively, the target grid may be projected at thesame time as the image 108. In this latter case, the target grid may beprojected using invisible wavelengths such as infrared light that thecamera 106 is capable of capturing so that the audience does not see thetarget grid during the presentation.

In FIG. 2, the exemplary device 102 has applied the projection outputcorrections discussed above according to various embodiments. Here, thedevice 102 projects the compensated projection image 108′ onto the samelocation where the uncompensated projected image 108 was projected. Theeffects of keystone, shifts, and streaks have been reduced so that thecompensated projection image 108′ more closely resembles a photograph,or a projection onto a more regular projection surface.

FIG. 3 shows an example where a projection device 202 is being used tocreate a larger projection onto a wall 208 or even onto a projectionscreen. The projection device 202 of this example may take many forms,such as the personal digital devices discussed above or a full scaleprojector. Here a person 210 is present between a projection output 204of the device 202 and the wall 208 where a projected image 212 isappearing. This is often the case during a presentation, such as wherethe person 210 interacts with the image 212 or merely walks by the wall208. Portions of the projected image 212 appear on the person 210, whichdistorts the appearance of the projected image 212. Furthermore, whenthe person 210 faces the audience and the projection output 204, thelight from the projection output 204 shines onto the face and eyes ofthe person 210, which results in discomfort.

According to exemplary embodiments, the device 202 includes a camera 206that captures the projected image 212 so that the captured image can becompared to the reference image to find distortions and compensate forthem. Using techniques discussed above, the device 202 may attempt tomodify the image to account for the change in depth of the field wherethe person 212 is present and to account for the variation in colors,intensities, and the like due to the colors and textures of the person'sclothing, skin, and hair. This may be done by capturing the projectedimage 212 with the camera 206. Alternatively, some of this distortionmay be captured by a target grid being projected and captured foranalysis, such as any keystone and any variation in the depth of thefield.

FIG. 4 shows an alternative approach to dealing with the distortioncaused by the person 210. Rather than compensating a projected image212′ to try to reduce or eliminate the distortion, it may be desirableto change the distortion. In this case, it may be desirable to project asilhouette of the person 210 in black that overlaps onto the person 210.In that case, to the audience it would appear that the projection isbeing generated between the person 210 and the wall 208 since theprojection image 212′ does not appear on the person 210. This may beless distracting for the audience. Another reason to project thesilhouette onto the person 210 is so that when the person 210 faces thedevice 202, no light from the projection output 204 would strike theperson's face and eyes so that the person 210 is not discomforted whenstanding in front of the projected image 212′.

FIG. 5 shows an example of a device 302 where a projection output 304projects an image 310. A camera 306 is present to capture the imageand/or a target grid if present. In order to manipulate the display, aperson places a hand 308 into the field of the projection output 304where the hand can be captured by the camera 306. The hand 308 maygesture in some recognizable manner, such as to point or even move in apattern. The camera 306 produces an image of the projected image 310 ortarget grid if present which the device 302 may then analyze. The device302 recognizes the gesture, either a static hand formation or a handmovement based on multiple frame captures by the camera 306. The device302 may then implement any image compensation associated with therecognized gesture.

FIG. 6 shows that the device 302 has recognized the gesture as being aquarter-clockwise rotation command. As such, the device 302 hasprojected a rotated image 310′ accordingly.

FIG. 7A shows components of an exemplary device for projecting acompensated image. The device includes a housing 403 within which thecomponents are located. Many different components may be includeddepending upon the desired functions of the device. FIG. 7A shows thosecomponents used during the projection compensation process, according toexemplary embodiments. However, it will be appreciated that othercomponents may also be included. For example, mobile phone componentsmay be included in addition to those shown.

As shown in FIG. 7A, the reference image data of memory location 402 isprovided to a projection output 404 and is accessed by a processor 412.A projected image 406 corresponding to the reference image data 402appears at the location where the projection output 404 is aimed. Acamera 408, also aimed at that location, captures an image of theprojected image 406 appearing at the location. The captured image datais stored in a memory location 410 where the captured image data isaccessed by the processor 412.

Upon the processor 412 having access to both the reference image dataand the captured image data, the processor 412 then computes thecorrections to be applied to produce compensated image data, inaccordance with exemplary embodiments. The processor 412 provides thecompensated image data to a memory location 414. The compensated imagedata is then provided as a signal to the projection output 404 so thatthe projected image 406 becomes the compensated image.

It will be appreciated that this feedback loop of the device of FIG. 7Amay operate a single time for a given session or may operatecontinuously. For example, if only those irregularities that will affectevery image the same are being corrected and they are static, such asthe angular relationship that results in keystoning, then the correctionmay be computed a single time and applied to different images. However,where the image to be projected changes or where the irregularities ofthe surface change over time during the session, then the correction maybe repeatedly calculated and applied so as to provide differentcorrections for different images and/or different irregularities of thesurface.

The processor 412 may be of various forms such as a general purposeprogrammable processor, an application specific processor, hardwireddigital logic, or various combinations thereof. The processor mayimplement logical operations to control the projection and capture ofimages and to compute the corrections to be applied.

The processor 412 and memory constituting the memory locations 402, 410,414 are examples of a computer readable media which store instructionsthat when performed implement various logical operations. Such computerreadable media may include various storage media including electronic,magnetic, and optical storage. Computer readable media may also includecommunications media, such as wired and wireless connections used totransfer the instructions or send and receive other data messages.

FIG. 7B shows components of another exemplary device for projecting acompensated image. The device includes a housing 503 within which thecomponents are located. FIG. 7B shows those components used during theprojection compensation process, according to exemplary embodiments.However, as with FIG. 7A, it will be appreciated that other componentsmay also be included. For example, mobile phone components may beincluded in addition to those shown.

As shown in FIG. 7B, a target grid output 508, such as a single ormultiple infrared projectors, projects a target grid 510 or othersimilar target onto a surface that will receive the projection. Datadefining the target grid 510 is also accessed by a processor 516. Acamera 512 that is aimed at that location where the target grid 510appears captures an image of the target grid 510 appearing at thelocation. The captured image data is stored in a memory location 514where the captured image data is accessed by the processor 516.

Upon the processor 516 also accessing the reference image data from amemory location 502, the processor 516 then computes the corrections tobe applied to produce compensated image data, in accordance withexemplary embodiments. The processor 516 provides the compensated imagedata to a memory location 518. The compensated image data is thenprovided to a projection output 504 so that a projected image 506 is thecompensated image. The target grid 510 is shown as being projected at alocation other than the location of the projected image 506 for clarityof illustration. It will be appreciated that the target grid 510 may beprojected onto the same location where the projected image 506 will beor is being projected. As discussed above, the target grid 510 may beprojected in infrared so that the target grid 510 may not be visible toan audience viewing the projected image 506 even though the target grid510 is projected onto the same location.

It will be appreciated that this feedback loop of the device of FIG. 7B,like that of FIG. 7A, may operate a single time for a given session ormay operate continuously. For example, if only those irregularities thatwill affect every image the same are being corrected and they arestatic, such as the angular relationship that results in keystoning,then the correction may be computed a single time and applied todifferent images. However, where the image to be projected changes orwhere the irregularities of the surface change over time during thesession, then the correction may be repeatedly calculated and applied soas to provide different corrections for different images and/ordifferent irregularities of the surface.

The processor 516, like the processor 412 of FIG. 7A, may be of variousforms such as a general purpose programmable processor, an applicationspecific processor, hardwired digital logic, or various combinationsthereof. The processor 516 may implement logical operations to controlthe projection of the image and the target grid, the capture of images,and to compute the corrections to be applied. The processor 516 andmemory constituting the memory locations 502, 514, and 518 are alsoexamples of a computer readable media

FIG. 8 shows a set of logical operations that may be performed by theprocessor 412 of FIG. 7A according to various embodiments. The processor412 obtains the reference image data at an image operation 802. Theprocessor 412 then produces a reference display signal for theprojection output 404 at a projection operation 804, such as byinteraction with a video adapter that converts the image data to asignal compatible with the projection output 404. The processor 412 mayinstruct the camera to obtain an image of the projected image appearingat the location at a capture operation 806.

Once the processor 412 has both the reference image and the capturedimage, the processor 412 then compares the two at a comparison operation808. Here the processor 412 may factor in any known keystone that thecamera perceives when attempting to detect keystoning of the projectedimage. Likewise, the processor 412 may apply the pixel by pixelcomparison here to detect color and intensity differences and the like,to find noise and/or patterns introduced by the surface receiving theprojection, to perceive objects between the projection output 404 andthe surface, and to recognize objects and gestures.

After determining the distortions present in the captured image, theprocessor 412 then computes the compensation data at a compensationoperation 810. Here the processor 412 determines the manipulations ofthe reference image data that are necessary so that when projected, thecompensated image will more closely match the reference image and/orhave different qualities such as containing a silhouette of anintervening person or object or be altered in correspondence with arecognized gesture. The generation of compensation data is discussedbelow in more detail in relation to FIG. 9.

After having generated the compensation data, the processor 412 thenapplies that compensation data to the reference image to produce acompensated image data at an application operation 812. The processor412 then produces a reference display signal for the projection output404 based on the compensated image data at a projection operation 814.

For embodiments where the compensation is continuous or at least goesthrough several iterations to increase the accuracy of the corrections,then the logical operations return to again capture an image of thecurrently corrected image or a next projected image at the captureoperation 806. For example, the surface may be in a first state duringthe current iteration of these logical operations but then change to asecond state during a subsequent iteration such that the compensation isdifferent. For example, the coin 112 of FIG. 1 may be removed or thedevice 102 may be re-positioned between iterations to present adifferent state of the surface 110. In this next iteration, differentimage data is captured and different compensation data is produced sothat a different compensated image results.

FIG. 9 shows one example of a set of logical operations that correspondto the compensation operation 810 of FIG. 8. The processor 412 mayprocess a series of different corrections for the image. One correctionmay assist in processing subsequent corrections such that thecorrections may be processed sequentially. For example, initiallycorrecting for keystoning may result in a more reliable pixel by pixelcomparison when correcting color and intensity distortions in theprojected image. FIG. 9 shows one exemplary sequence to the corrections,but it will be appreciated that the sequence may be changed to manydifferent orders of corrections. Furthermore, the processor 412 maycalculate one or more corrections contemporaneously rather thansequentially.

At a query operation 902, the processor 412 detects whether theprojected image 406 is keystoned. If the projected image 406 iskeystoned, then the processor 412 calculates the amount of warp that ispresent and calculates an amount of compensating warp at a calculationoperation 904. Here, the amount of keystone in the captured image thatis the result of the camera 408 viewing the projected image 406 atessentially the same angle as the projection output 404 is factored intothe calculation of the warp. The warp correction may be calculated toproduce a captured image having a keystone that matches the keystoneexpected to be present due to the angle of the camera 408 to thesurface.

If there is no keystone present or after the keystone calculations arecomplete, the processor 412 then detects whether the colors match ateach pixel location at a query operation 906. If the colors do not matchat each pixel location, then the processor 412 calculates the colordifferences and from that calculates the color correction to be appliedto negate the color differences at a calculation operation 908.

If there is no color mismatch present or after the color mismatchcalculations are complete, the processor 412 then detects whether arecognizable object is present in the field of view captured by thecamera 408 at a query operation 910. For example, the processor 412 mayrecognize a hand or torso of a person based on the distortion that ispresent in the captured image. The processor 412 may maintain a libraryof object shapes in memory and associate the shapes with a correction.For example, when a torso is recognized, the correction may be toproject a silhouette by utilizing black for those pixels where the torsois present in the captured image. The processor 412 determines theappropriate correction for the recognized object at a calculationoperation 912.

If there is no recognized object or after the correction for arecognized object has been completed, the processor 412 then detectswhether a planar disturbance is present at a query operation 914. Forexample, the coin 112 of FIG. 1 presents a planar disturbance. Otherplanar disturbances might include a surface receiving the projectionthat is inherently non-planar, such as a convex or concave surface. Theprocessor 412 attempts to correct for the planar disturbance by shiftingpixels to create intentional overlaps of pixels or to create intentionalempty areas between pixels at a calculation operation 916.

If there is no planar disturbance or after the planar disturbancecalculations are complete, the processor 412 then detects whether arecognized gesture is present at a query operation 918. The gesture maybe static, such as a certain positioning of a person's fingers, or maybe dynamic by movement of a person's hand. Where dynamic, the processor412 may observe multiple image captures and compare one to the next todetect the motion. The processor 412 may have access to a library ofgestures and associated actions to be taken as a correction to theprojected image. Upon recognizing a gesture, the processor 412 appliesan image manipulation that is associated with the gesture at acalculation operation 920.

FIG. 10 shows another set of logical operations that may be performed,but by the processor 516 of FIG. 7B according to various embodiments.The processor 516 instructs the target grid output 508 to project thetarget grid at a target operation 1002. The processor 516 may theninstruct the camera 512 to obtain an image of the target appearing atthe location where the projected image is or will be at a captureoperation 1004.

The processor 516 has access to the reference target grid beingprojected and compares the coordinates of the reference target grid tothose of the target grid in the captured image at a comparison operation1006. Thus, rather than doing a pixel by pixel analysis, the processor516 performs a grid point by grid point analysis of the target grid,where that target grid may have a grid point resolution that is greaterthan or less than the resolution of the reference image. From thisanalysis, the processor 516 determines the irregularities present at thesurface relative to the perspective of the camera 512.

After having compared the grid points of the target grid, the processor516 then generates the compensation data needed to account fordistortions that are likely to occur in a projected image at acompensation operation 1008. The processor 516 then applies thecompensation data to the reference image to be projected at anapplication operation 1010 to produce a compensated image data. Theprocessor 516 then provides the compensated image data where videohardware produces a corresponding compensated display signal that theprojection output 504 projects as the compensated projection image 506at a calculation operation 1012.

For embodiments where the compensation is continuous or at least goesthrough several iterations to increase the accuracy of the corrections,then the logical operations return to again capture an image of thetarget grid at the capture operation 1004. For example, the surface maybe in a first state during the current iteration of these logicaloperations but then change to a second state during a subsequentiteration such that the compensation is different. As in the examplediscussed above in relation to FIG. 8, the coin 112 of FIG. 1 may beremoved or the device 102 may be re-positioned between iterations topresent a different state of the surface 110. In this next iteration,different image data is captured and different compensation data isproduced so that a different compensated image results.

Thus, by capturing images of the location where images are being or willbe projected, compensation for irregularities may be determined andapplied. Distortions otherwise caused by irregularities of the surfacereceiving the projection may be reduced or changed, depending upon whatis desired. The projected image may be a better representation of thesource image as a result.

While embodiments have been particularly shown and described, it will beunderstood by those skilled in the art that various other changes in theform and details may be made therein without departing from the spiritand scope of the invention.

1. A device that projects an image, comprising: a housing; a projectionoutput within the housing producing a projected output that extends to afirst position beyond the housing; a camera within the housing andhaving a fixed relationship relative to the projection output, thecamera capturing an image from the first position; and a processorwithin the housing and in communication with the projection output andthe camera, the processor providing a source image to the projectionoutput, the processor receiving the captured image from the camera, theprocessor comparing the captured image relative to the source image inrelation to the fixed relationship between the camera and the projectionoutput to determine at least one difference, the processor creating animage based on the at least one difference, and the processor providingthe created image to the projection output in placed of the sourceimage.
 2. The device of claim 1, wherein the processor creates the imagebased on the at least one difference by applying a compensation toreduce the difference.
 3. The device of claim 2, wherein the differencecomprises artifacts resulting from characteristics of a surface at thefirst position and wherein the compensation reduces effects of thecharacteristics of the surface.
 4. The device of claim 2, wherein thedifference comprises a keystone due to an angular relationship between asurface at the first position and the projection output and wherein thecompensation reduces effects of the keystone.
 5. The device of claim 2,wherein the difference comprises an object between a surface at thefirst position and the projection output and wherein the compensationreduces the effects of the object.
 6. The device of claim 1, wherein theprocessor creates the image based on the at least one difference byapplying a compensation to change the difference.
 7. The device of claim6, wherein the difference comprises an object between a surface at thefirst position and the projection output and wherein the compensationchanges the difference by omitting from the projected output a portionof the projected output that strikes the object.
 8. The device of claim2, wherein the differences comprises a gesture by a recognized objectpresent between the first position and the projection output, andwherein the compensation comprises manipulating the image in accordancewith a pre-defined and stored scheme corresponding to the gesture.
 9. Acomputer readable medium containing instructions that perform actscomprising: projecting a reference target onto a first location;capturing image data of the first location while the target is beingprojected onto the first location; comparing the captured image data tothe reference target to detect at least one difference; generatingcompensation data based on the at least one difference; applying thecompensation data to image data to produce compensated image data; andprojecting a compensated image corresponding to the compensated imagedata onto the first location.
 10. The computer readable medium of claim9, applying the compensation data to the image data produces acompensated image data that reduces the at least one difference.
 11. Thecomputer readable medium of claim 10, wherein the difference comprises akeystone due to an angular relationship between a surface at the firstposition and a projection output and wherein applying the compensationdata reduces effects of the keystone.
 12. The computer readable mediumof claim 10, wherein the difference comprises an object between asurface at the first position and a projection output and whereinapplying the compensation data reduces the effects of the object. 13.The computer readable medium of claim 9, wherein applying thecompensation data to the reference image produces a compensated imagedata that changes the at least one difference.
 14. The computer readablemedium of claim 13, wherein the difference comprises an object between asurface at the first position and a projection output and whereinapplying the compensation data omits from the projected output a portionof the projected output that strikes the object.
 15. The computerreadable medium of claim 9, wherein the target is projected withinfrared light while an image corresponding to the image data is beingprojected onto the first location.
 16. A method of projecting an image,comprising: obtaining reference image data; producing a referencedisplay signal from the reference image data, the reference displaysignal being projected onto a first location, the first location being adynamic surface; capturing first image data of the first location whilethe reference display signal is being projected onto the first locationwhile the first location is in a first state; comparing the capturedfirst image data to the reference image data to detect at least onefirst difference; generating first compensation data based on the atleast one first difference; applying the first compensation data to thereference image data to produce compensated first image data; producinga first compensated display signal from the compensated image data, thefirst compensated display signal being projected onto the firstlocation; capturing second image data of the first location while thefirst compensated display signal is being projected onto the firstlocation while the first location is in a second state different thanthe first state; comparing the captured second image data to thereference image data to detect at least one second difference;generating second compensation data based on the at least one seconddifference; applying the second compensation data to the reference imagedata to produce compensated second image data; and producing a secondcompensated display signal from the compensated second image data, thesecond compensated display signal being projected onto the firstlocation.
 17. The method of claim 16, wherein the first and seconddifferences each comprise at least one of an angular relationship of asurface at the first position to a projection output producing akeystone; color variations, patterns, and visual noise in the surface atthe first position producing unintended color variations, patterns, andvisual noise in the appearance of the reference display signal beingprojected and wherein applying the first and second compensation datareduces the appearance of the keystone relationship, color variations,patterns, and visual noise.
 18. The method of claim 16, wherein thedifference comprises a person between the first surface and a projectionoutput and wherein applying the first and second compensation dataresults in projecting a blank silhouette onto to the person whileprojecting a remaining portion of the first and second compensateddisplay signals onto the first location.
 19. The method of claim 16,wherein the difference comprises a gesture by a recognizable object andwherein applying the first and second compensation data results inprojecting the first and second compensated display signal to produce arotated image.
 20. The method of claim 16, wherein the dynamic surfaceis non-planar.